<!DOCTYPE html>
<html >

<head>

  <meta charset="UTF-8">
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
  <title>Linux学习</title>
  <meta name="description" content="Linux学习">
  <meta name="generator" content="bookdown 0.5 and GitBook 2.6.7">

  <meta property="og:title" content="Linux学习" />
  <meta property="og:type" content="book" />
  
  
  
  

  <meta name="twitter:card" content="summary" />
  <meta name="twitter:title" content="Linux学习" />
  
  
  

<meta name="author" content="train">


<meta name="date" content="2018-03-23">

  <meta name="viewport" content="width=device-width, initial-scale=1">
  <meta name="apple-mobile-web-app-capable" content="yes">
  <meta name="apple-mobile-web-app-status-bar-style" content="black">
  
  
<link rel="prev" href="softinstall.html">
<link rel="next" href="sed.html">
<script src="libs/jquery-2.2.3/jquery.min.js"></script>
<link href="libs/gitbook-2.6.7/css/style.css" rel="stylesheet" />
<link href="libs/gitbook-2.6.7/css/plugin-bookdown.css" rel="stylesheet" />
<link href="libs/gitbook-2.6.7/css/plugin-highlight.css" rel="stylesheet" />
<link href="libs/gitbook-2.6.7/css/plugin-search.css" rel="stylesheet" />
<link href="libs/gitbook-2.6.7/css/plugin-fontsettings.css" rel="stylesheet" />









<style type="text/css">
div.sourceCode { overflow-x: auto; }
table.sourceCode, tr.sourceCode, td.lineNumbers, td.sourceCode {
  margin: 0; padding: 0; vertical-align: baseline; border: none; }
table.sourceCode { width: 100%; line-height: 100%; }
td.lineNumbers { text-align: right; padding-right: 4px; padding-left: 4px; color: #aaaaaa; border-right: 1px solid #aaaaaa; }
td.sourceCode { padding-left: 5px; }
code > span.kw { color: #007020; font-weight: bold; } /* Keyword */
code > span.dt { color: #902000; } /* DataType */
code > span.dv { color: #40a070; } /* DecVal */
code > span.bn { color: #40a070; } /* BaseN */
code > span.fl { color: #40a070; } /* Float */
code > span.ch { color: #4070a0; } /* Char */
code > span.st { color: #4070a0; } /* String */
code > span.co { color: #60a0b0; font-style: italic; } /* Comment */
code > span.ot { color: #007020; } /* Other */
code > span.al { color: #ff0000; font-weight: bold; } /* Alert */
code > span.fu { color: #06287e; } /* Function */
code > span.er { color: #ff0000; font-weight: bold; } /* Error */
code > span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } /* Warning */
code > span.cn { color: #880000; } /* Constant */
code > span.sc { color: #4070a0; } /* SpecialChar */
code > span.vs { color: #4070a0; } /* VerbatimString */
code > span.ss { color: #bb6688; } /* SpecialString */
code > span.im { } /* Import */
code > span.va { color: #19177c; } /* Variable */
code > span.cf { color: #007020; font-weight: bold; } /* ControlFlow */
code > span.op { color: #666666; } /* Operator */
code > span.bu { } /* BuiltIn */
code > span.ex { } /* Extension */
code > span.pp { color: #bc7a00; } /* Preprocessor */
code > span.at { color: #7d9029; } /* Attribute */
code > span.do { color: #ba2121; font-style: italic; } /* Documentation */
code > span.an { color: #60a0b0; font-weight: bold; font-style: italic; } /* Annotation */
code > span.cv { color: #60a0b0; font-weight: bold; font-style: italic; } /* CommentVar */
code > span.in { color: #60a0b0; font-weight: bold; font-style: italic; } /* Information */
</style>

<link rel="stylesheet" href="style.css" type="text/css" />
</head>

<body>



  <div class="book without-animation with-summary font-size-2 font-family-1" data-basepath=".">

    <div class="book-summary">
      <nav role="navigation">

<ul class="summary">
<li><a href="http://www.ehbio.com"><img src="http://www.ehbio.com/logos/ehbio_gitbook_logo.png" width="95%"></a></li>

<li class="divider"></li>
<li class="chapter" data-level="" data-path="index.html"><a href="index.html"><i class="fa fa-check"></i>EHBIO Gene Technology</a></li>
<li class="chapter" data-level="1" data-path="linux-basic.html"><a href="linux-basic.html"><i class="fa fa-check"></i><b>1</b> Linux基础内容</a><ul>
<li class="chapter" data-level="1.1" data-path="linux-basic.html"><a href="linux-basic.html#basicDir"><i class="fa fa-check"></i><b>1.1</b> Linux系统简介和目录理解</a><ul>
<li class="chapter" data-level="1.1.1" data-path="linux-basic.html"><a href="linux-basic.html#linux"><i class="fa fa-check"></i><b>1.1.1</b> 为什么要用Linux系统</a></li>
<li class="chapter" data-level="1.1.2" data-path="linux-basic.html"><a href="linux-basic.html#linux"><i class="fa fa-check"></i><b>1.1.2</b> Linux系统简介</a></li>
<li class="chapter" data-level="1.1.3" data-path="linux-basic.html"><a href="linux-basic.html#linux"><i class="fa fa-check"></i><b>1.1.3</b> 如何获取Linux系统</a></li>
<li class="chapter" data-level="1.1.4" data-path="linux-basic.html"><a href="linux-basic.html#linux"><i class="fa fa-check"></i><b>1.1.4</b> 初识Linux系统</a></li>
<li class="chapter" data-level="1.1.5" data-path="linux-basic.html"><a href="linux-basic.html#section-1.1.5"><i class="fa fa-check"></i><b>1.1.5</b> 我的电脑在哪？</a></li>
<li class="chapter" data-level="1.1.6" data-path="linux-basic.html"><a href="linux-basic.html#section-1.1.6"><i class="fa fa-check"></i><b>1.1.6</b> 硬件信息查看</a></li>
<li class="chapter" data-level="1.1.7" data-path="linux-basic.html"><a href="linux-basic.html#section-1.1.7"><i class="fa fa-check"></i><b>1.1.7</b> 目录内容查看</a></li>
<li class="chapter" data-level="1.1.8" data-path="linux-basic.html"><a href="linux-basic.html#section-1.1.8"><i class="fa fa-check"></i><b>1.1.8</b> 新建目录</a></li>
<li class="chapter" data-level="1.1.9" data-path="linux-basic.html"><a href="linux-basic.html#section-1.1.9"><i class="fa fa-check"></i><b>1.1.9</b> 访问文件</a></li>
<li class="chapter" data-level="1.1.10" data-path="linux-basic.html"><a href="linux-basic.html#section-1.1.10"><i class="fa fa-check"></i><b>1.1.10</b> 获取可用命令行参数</a></li>
<li class="chapter" data-level="1.1.11" data-path="linux-basic.html"><a href="linux-basic.html#a"><i class="fa fa-check"></i><b>1.1.11</b> 小结</a></li>
<li class="chapter" data-level="1.1.12" data-path="linux-basic.html"><a href="linux-basic.html#section-1.1.12"><i class="fa fa-check"></i><b>1.1.12</b> 练习题</a></li>
</ul></li>
<li class="chapter" data-level="1.2" data-path="linux-basic.html"><a href="linux-basic.html#fileoperation"><i class="fa fa-check"></i><b>1.2</b> Linux下文件操作</a><ul>
<li class="chapter" data-level="1.2.1" data-path="linux-basic.html"><a href="linux-basic.html#section-1.2.1"><i class="fa fa-check"></i><b>1.2.1</b> 文件上下翻转和左右翻转</a></li>
<li class="chapter" data-level="1.2.2" data-path="linux-basic.html"><a href="linux-basic.html#section-1.2.2"><i class="fa fa-check"></i><b>1.2.2</b> 其它新建文件的方式</a></li>
<li class="chapter" data-level="1.2.3" data-path="linux-basic.html"><a href="linux-basic.html#section-1.2.3"><i class="fa fa-check"></i><b>1.2.3</b> 文件拷贝、移动、重命名</a></li>
<li class="chapter" data-level="1.2.4" data-path="linux-basic.html"><a href="linux-basic.html#linux"><i class="fa fa-check"></i><b>1.2.4</b> Linux下命令的一些异常情况</a></li>
<li class="chapter" data-level="1.2.5" data-path="linux-basic.html"><a href="linux-basic.html#linux"><i class="fa fa-check"></i><b>1.2.5</b> Linux下文件内容操作</a></li>
<li class="chapter" data-level="1.2.6" data-path="linux-basic.html"><a href="linux-basic.html#section-1.2.6"><i class="fa fa-check"></i><b>1.2.6</b> 小结和练习</a></li>
</ul></li>
<li class="chapter" data-level="1.3" data-path="linux-basic.html"><a href="linux-basic.html#shortcut"><i class="fa fa-check"></i><b>1.3</b> Linux终端常用快捷操作</a></li>
<li class="chapter" data-level="1.4" data-path="linux-basic.html"><a href="linux-basic.html#stdinoutpipe"><i class="fa fa-check"></i><b>1.4</b> Linux下的标准输入、输出、重定向、管道</a></li>
<li class="chapter" data-level="1.5" data-path="linux-basic.html"><a href="linux-basic.html#filecontent"><i class="fa fa-check"></i><b>1.5</b> Linux文件内容操作</a><ul>
<li class="chapter" data-level="1.5.1" data-path="linux-basic.html"><a href="linux-basic.html#section-1.5.1"><i class="fa fa-check"></i><b>1.5.1</b> 文件生成</a></li>
<li class="chapter" data-level="1.5.2" data-path="linux-basic.html"><a href="linux-basic.html#section-1.5.2"><i class="fa fa-check"></i><b>1.5.2</b> 文件排序</a></li>
<li class="chapter" data-level="1.5.3" data-path="linux-basic.html"><a href="linux-basic.html#fasta"><i class="fa fa-check"></i><b>1.5.3</b> FASTA序列提取</a></li>
</ul></li>
<li class="chapter" data-level="1.6" data-path="linux-basic.html"><a href="linux-basic.html#section-1.6"><i class="fa fa-check"></i><b>1.6</b> 命令运行监测</a></li>
<li class="chapter" data-level="1.7" data-path="linux-basic.html"><a href="linux-basic.html#references"><i class="fa fa-check"></i><b>1.7</b> References</a></li>
</ul></li>
<li class="chapter" data-level="2" data-path="softinstall.html"><a href="softinstall.html"><i class="fa fa-check"></i><b>2</b> Linux下软件安装相关</a><ul>
<li class="chapter" data-level="2.1" data-path="softinstall.html"><a href="softinstall.html#fileattributeplusX"><i class="fa fa-check"></i><b>2.1</b> 文件属性和可执行属性</a><ul>
<li class="chapter" data-level="2.1.1" data-path="softinstall.html"><a href="softinstall.html#fileattribute"><i class="fa fa-check"></i><b>2.1.1</b> 文件属性</a></li>
<li class="chapter" data-level="2.1.2" data-path="softinstall.html"><a href="softinstall.html#executable"><i class="fa fa-check"></i><b>2.1.2</b> 可执行属性</a></li>
</ul></li>
<li class="chapter" data-level="2.2" data-path="softinstall.html"><a href="softinstall.html#path"><i class="fa fa-check"></i><b>2.2</b> 环境变量</a><ul>
<li class="chapter" data-level="2.2.1" data-path="softinstall.html"><a href="softinstall.html#section-2.2.1"><i class="fa fa-check"></i><b>2.2.1</b> 环境变量的补充</a></li>
<li class="chapter" data-level="2.2.2" data-path="softinstall.html"><a href="softinstall.html#-1"><i class="fa fa-check"></i><b>2.2.2</b> 小结和练习</a></li>
</ul></li>
<li class="chapter" data-level="2.3" data-path="softinstall.html"><a href="softinstall.html#softInstallways"><i class="fa fa-check"></i><b>2.3</b> 软件安装的几种方式</a><ul>
<li class="chapter" data-level="2.3.1" data-path="softinstall.html"><a href="softinstall.html#section-2.3.1"><i class="fa fa-check"></i><b>2.3.1</b> 系统包管理器安装</a></li>
<li class="chapter" data-level="2.3.2" data-path="softinstall.html"><a href="softinstall.html#section-2.3.2"><i class="fa fa-check"></i><b>2.3.2</b> 下载二进制文件</a></li>
<li class="chapter" data-level="2.3.3" data-path="softinstall.html"><a href="softinstall.html#section-2.3.3"><i class="fa fa-check"></i><b>2.3.3</b> 源码编译安装</a></li>
<li class="chapter" data-level="2.3.4" data-path="softinstall.html"><a href="softinstall.html#pythonpac"><i class="fa fa-check"></i><b>2.3.4</b> Python包的安装</a></li>
<li class="chapter" data-level="2.3.5" data-path="softinstall.html"><a href="softinstall.html#anaconda"><i class="fa fa-check"></i><b>2.3.5</b> Anaconda的两个福利</a></li>
<li class="chapter" data-level="2.3.6" data-path="softinstall.html"><a href="softinstall.html#rr"><i class="fa fa-check"></i><b>2.3.6</b> R和R包的安装</a></li>
</ul></li>
<li class="chapter" data-level="2.4" data-path="softinstall.html"><a href="softinstall.html#condaInstall"><i class="fa fa-check"></i><b>2.4</b> Conda安装</a><ul>
<li class="chapter" data-level="2.4.1" data-path="softinstall.html"><a href="softinstall.html#condaInstallConfig"><i class="fa fa-check"></i><b>2.4.1</b> Conda安装和配置</a></li>
<li class="chapter" data-level="2.4.2" data-path="softinstall.html"><a href="softinstall.html#conda"><i class="fa fa-check"></i><b>2.4.2</b> Conda基本使用</a></li>
<li class="chapter" data-level="2.4.3" data-path="softinstall.html"><a href="softinstall.html#condachannel"><i class="fa fa-check"></i><b>2.4.3</b> Conda的channel</a></li>
<li class="chapter" data-level="2.4.4" data-path="softinstall.html"><a href="softinstall.html#section-2.4.4"><i class="fa fa-check"></i><b>2.4.4</b> 创建不同的软件运行环境</a></li>
<li class="chapter" data-level="2.4.5" data-path="softinstall.html"><a href="softinstall.html#condar"><i class="fa fa-check"></i><b>2.4.5</b> Conda配置R</a></li>
<li class="chapter" data-level="2.4.6" data-path="softinstall.html"><a href="softinstall.html#conda"><i class="fa fa-check"></i><b>2.4.6</b> Conda环境简化运行</a></li>
</ul></li>
<li class="chapter" data-level="2.5" data-path="softinstall.html"><a href="softinstall.html#makefile"><i class="fa fa-check"></i><b>2.5</b> Makefile知识</a><ul>
<li class="chapter" data-level="2.5.1" data-path="softinstall.html"><a href="softinstall.html#references-1"><i class="fa fa-check"></i><b>2.5.1</b> References</a></li>
</ul></li>
</ul></li>
<li class="chapter" data-level="3" data-path="awk.html"><a href="awk.html"><i class="fa fa-check"></i><b>3</b> awk命令</a><ul>
<li class="chapter" data-level="3.1" data-path="awk.html"><a href="awk.html#awk"><i class="fa fa-check"></i><b>3.1</b> awk基本参数解释</a></li>
<li class="chapter" data-level="3.2" data-path="awk.html"><a href="awk.html#section-3.2"><i class="fa fa-check"></i><b>3.2</b> 常见操作</a></li>
<li class="chapter" data-level="3.3" data-path="awk.html"><a href="awk.html#section-3.3"><i class="fa fa-check"></i><b>3.3</b> 糅合操作</a></li>
<li class="chapter" data-level="3.4" data-path="awk.html"><a href="awk.html#section-3.4"><i class="fa fa-check"></i><b>3.4</b> 生信宝典</a></li>
<li class="chapter" data-level="3.5" data-path="awk.html"><a href="awk.html#rpythoncytoscape"><i class="fa fa-check"></i><b>3.5</b> 生信宝典，生物信息学习系列教程，转录组，宏基因组，外显子组，R作图，Python学习，Cytoscape视频教程</a></li>
<li class="chapter" data-level="3.6" data-path="awk.html"><a href="awk.html#section-3.6"><i class="fa fa-check"></i><b>3.6</b> 生信宝典，最好的生物信息培训课程，培训课程资料</a></li>
</ul></li>
<li class="chapter" data-level="4" data-path="sed.html"><a href="sed.html"><i class="fa fa-check"></i><b>4</b> SED命令</a><ul>
<li class="chapter" data-level="4.1" data-path="sed.html"><a href="sed.html#sed"><i class="fa fa-check"></i><b>4.1</b> sed基本参数解释</a></li>
<li class="chapter" data-level="4.2" data-path="softinstall.html"><a href="softinstall.html#-1"><i class="fa fa-check"></i><b>4.2</b> 常见操作</a></li>
<li class="chapter" data-level="4.3" data-path="sed.html"><a href="sed.html#rpythoncytoscape-1"><i class="fa fa-check"></i><b>4.3</b> 生信宝典，生物信息学习系列教程，转录组，宏基因组，外显子组，R作图，Python学习，Cytoscape视频教程</a></li>
<li class="chapter" data-level="4.4" data-path="softinstall.html"><a href="softinstall.html#-1"><i class="fa fa-check"></i><b>4.4</b> 生信宝典，最好的生物信息培训课程，培训课程资料</a></li>
</ul></li>
<li class="chapter" data-level="5" data-path="bash-string.html"><a href="bash-string.html"><i class="fa fa-check"></i><b>5</b> Bash 字符串处理</a><ul>
<li class="chapter" data-level="5.1" data-path="bash-string.html"><a href="bash-string.html#bash"><i class="fa fa-check"></i><b>5.1</b> Bash特殊字符</a></li>
<li class="chapter" data-level="5.2" data-path="bash-string.html"><a href="bash-string.html#bash"><i class="fa fa-check"></i><b>5.2</b> Bash变量</a></li>
<li class="chapter" data-level="5.3" data-path="bash-string.html"><a href="bash-string.html#bash"><i class="fa fa-check"></i><b>5.3</b> Bash操作符</a></li>
<li class="chapter" data-level="5.4" data-path="bash-string.html"><a href="bash-string.html#shelltest"><i class="fa fa-check"></i><b>5.4</b> Shell中条件和test命令</a></li>
<li class="chapter" data-level="5.5" data-path="bash-string.html"><a href="bash-string.html#shell"><i class="fa fa-check"></i><b>5.5</b> Shell流控制</a></li>
<li class="chapter" data-level="5.6" data-path="bash-string.html"><a href="bash-string.html#shell"><i class="fa fa-check"></i><b>5.6</b> Shell函数</a></li>
<li class="chapter" data-level="5.7" data-path="bash-string.html"><a href="bash-string.html#section-5.7"><i class="fa fa-check"></i><b>5.7</b> 输入输出</a></li>
<li class="chapter" data-level="5.8" data-path="bash-string.html"><a href="bash-string.html#-"><i class="fa fa-check"></i><b>5.8</b> 命令行处理 命令行处理命令:</a></li>
<li class="chapter" data-level="5.9" data-path="bash-string.html"><a href="bash-string.html#section-5.9"><i class="fa fa-check"></i><b>5.9</b> 进程和作业控制</a></li>
</ul></li>
<li class="chapter" data-level="6" data-path="others.html"><a href="others.html"><i class="fa fa-check"></i><b>6</b> 其它</a></li>
<li class="chapter" data-level="7" data-path="bioinfo-questions.html"><a href="bioinfo-questions.html"><i class="fa fa-check"></i><b>7</b> Bioinfo questions</a></li>
<li class="chapter" data-level="8" data-path="supplemental.html"><a href="supplemental.html"><i class="fa fa-check"></i><b>8</b> Supplemental</a></li>
<li class="divider"></li>
<li><a href="mailto:ct@ehbio.com" target="blank">ct@ehbio.com</a></li>

</ul>

      </nav>
    </div>

    <div class="book-body">
      <div class="body-inner">
        <div class="book-header" role="navigation">
          <h1>
            <i class="fa fa-circle-o-notch fa-spin"></i><a href="./">Linux学习</a>
          </h1>
        </div>

        <div class="page-wrapper" tabindex="-1" role="main">
          <div class="page-inner">

            <section class="normal" id="section-">
<div id="awk" class="section level1">
<h1><span class="header-section-number">3</span> awk命令</h1>
<p>在<a href="http://mp.weixin.qq.com/s?__biz=MzI5MTcwNjA4NQ==&amp;mid=2247483823&amp;idx=1&amp;sn=ac62450f0475dc9513e75009f0670f45&amp;chksm=ec0dc425db7a4d3300f547caeaee709425dd0a41c86be18aab44d41619a4d18944289b0deaf8#rd">文件排序和FASTA文件操作</a>中简述了<code>awk</code>和<code>sed</code>的使用，作为一个引子。本篇则详细列举关于<code>awk</code>常用的操作和一些偏门的操作。</p>
<div id="awk" class="section level2">
<h2><span class="header-section-number">3.1</span> awk基本参数解释</h2>
<p>awk擅长于对文件按行操作，每次读取一行，然后进行相应的操作。</p>
<p>awk读取单个文件时的基本语法格式是<code>awk 'BEGIN{OFS=FS=&quot;\t&quot;}{print $0, $1;}' filename</code>。</p>
<p>读取多个文件时的语法是<code>awk 'BEGIN{OFS=FS=&quot;\t&quot;}ARGIND==1{print $0, $1;}ARGIND==2{}' file1 file2</code>。</p>
<p>awk后面的命令部分是用引号括起来的，可以单引号，可以双引号，但注意不能与内部命令中用到的引号相同，否则会导致最相邻的引号视为一组，引发解释错误。</p>
<p><code>OFS</code>: 文件输出时的列分隔符 (output field separtor)</p>
<p><code>FS</code>: 文件输入时的列分隔符 (field separtor)</p>
<p><code>BEGIN</code>: 设置初始参数，初始化变量</p>
<p><code>END</code>: 读完文件后做最终的处理</p>
<p>其它<code>{}</code>：循环读取文件的每一行</p>
<p><code>$0</code>表示一行内容；<code>$1</code>, <code>$2</code>, … <code>$NF</code>表示第一列，第二列到最后一列。</p>
<p><code>NF (number of fields)</code>文件多少列；<code>NR (number of rows)</code> 文件读了多少行: <code>FNR</code> 当前文件读了多少行，常用于多文件操作时。</p>
<p><code>a[$1]=1</code>: 索引操作，类似于python中的字典，在<code>ID map</code>，<code>统计</code>中有很多应用。</p>
</div>
<div id="section-3.2" class="section level2">
<h2><span class="header-section-number">3.2</span> 常见操作</h2>
<ul>
<li>针对特定列的计算，比如wig文件的标准化</li>
</ul>
<div class="sourceCode"><pre class="sourceCode bash"><code class="sourceCode bash"><span class="kw">ct@ehbio</span>:~/sxbd$ cat ehbio.wig
<span class="kw">variableStep</span> chrom=chr2
<span class="kw">300701</span>  12.5
<span class="kw">300702</span>  12.5
<span class="kw">300703</span>  12.5
<span class="kw">300704</span>  12.5
<span class="kw">300705</span>  12.5
<span class="kw">ct@ehbio</span>:~/sxbd$ awk <span class="st">&#39;BEGIN{OFS=FS=&quot;\t&quot;}{$2=$2*10^6/(2.5*10^6); print $0}&#39;</span> wig 
<span class="kw">tep</span> chrom=chr2  0
<span class="kw">300701</span>  4.4
<span class="kw">300702</span>  4.8
<span class="kw">300703</span>  4
<span class="kw">300704</span>  4.8
<span class="kw">300705</span>  4.8</code></pre></div>
<ul>
<li>计算某列内容出现的次数。</li>
</ul>
<div class="sourceCode"><pre class="sourceCode bash"><code class="sourceCode bash"><span class="kw">ct@ehbio</span>:~/sxbd$ cat count 
<span class="kw">ID</span>  Type
<span class="kw">Pou5f1</span>  Pluripotency
<span class="kw">Nanog</span>   Pluripotency
<span class="kw">Sox2</span>    Neuron
<span class="kw">Tet1</span>    Epigenetic
<span class="kw">Tet3</span>    Epigenetic
<span class="kw">Myc</span> Oncogene
<span class="kw">ct@ehbio</span>:~/sxbd$ awk <span class="st">&#39;BEGIN{OFS=FS=&quot;\t&quot;}{if(FNR&gt;1) a[$2]+=1;}END{print &quot;Type\tCount&quot;; for(i in a) print i,a[i];}&#39;</span> count
<span class="kw">Type</span>    Count
<span class="kw">Neuron</span>  1
<span class="kw">Epigenetic</span>  2
<span class="kw">Oncogene</span>    1
<span class="kw">Pluripotency</span>    2

<span class="co"># 这个也可以用下面方式代替，但不直接</span>
<span class="kw">ct@ehbio</span>:~/sxbd$ tail -n +2 count <span class="kw">|</span> <span class="kw">cut</span> -f 2 <span class="kw">|</span> <span class="kw">sort</span> <span class="kw">|</span> <span class="kw">uniq</span> -c <span class="kw">|</span> <span class="kw">sed</span> -e <span class="st">&#39;s/^  *//&#39;</span> -e <span class="st">&#39;s/  */\t/&#39;</span> 
<span class="kw">2</span>   Epigenetic
<span class="kw">1</span>   Neuron
<span class="kw">1</span>   Oncogene
<span class="kw">2</span>   Pluripotency</code></pre></div>
<ul>
<li>之前也提到过的<a href="http://mp.weixin.qq.com/s/rZ26i19hiS5ZOqIoqkL1Wg">列操作</a>，从GTF文件中提取启动子区域</li>
</ul>
<div class="sourceCode"><pre class="sourceCode bash"><code class="sourceCode bash"><span class="kw">sed</span> <span class="st">&#39;s/&quot;/\t/g&#39;</span> GRCh38.gtf <span class="kw">|</span> <span class="kw">awk</span> <span class="st">&#39;BEGIN{OFS=FS=&quot;\t&quot;}{if($3==&quot;gene&quot;) {ensn=$10; symbol=$16; if($7==&quot;+&quot;) {start=$4-1; up=start-1000; if(up&lt;0) up=0; dw=start+500; print $1,up, dw, ensn, symbol, $7;} else if($7==&quot;-&quot;) {start=$5-1; up=start+1000; dw=start-500; if(dw&lt;0) dw=0; print $1,dw,up,ensn,symbol,$7}}}&#39;</span> <span class="kw">|</span> <span class="kw">sort</span> -k1,1 -k2,2n <span class="kw">&gt;</span>GRCh38.promoter.bed</code></pre></div>
<ul>
<li>数据矩阵的格式化输出</li>
</ul>
<div class="sourceCode"><pre class="sourceCode bash"><code class="sourceCode bash"><span class="kw">ct@ehbio</span>:~/sxbd$ cat numertic.matrix 
<span class="kw">ID</span>  A   B   C
<span class="kw">a</span>   1.002   1.234   1.999
<span class="kw">b</span>   2.333   4.232   0.889
<span class="kw">ct@ehbio</span>:~/sxbd$ awk <span class="st">&#39;{if(FNR==1) print $0; else {printf &quot;%s%s&quot;,$1,FS; for (i=2; i&lt;=NF; i++) printf &quot;%.1f %s&quot;, $i, (i==NF?RS:FS)}}&#39;</span> numertic.matrix 
<span class="kw">ID</span>  A   B   C
<span class="kw">a</span> 1.0  1.2  2.0 
<span class="kw">b</span> 2.3  4.2  0.9 </code></pre></div>
<ul>
<li>判断FASTQ文件中，输出质量值的长度是与序列长度不一致的序列ID</li>
</ul>
<div class="sourceCode"><pre class="sourceCode bash"><code class="sourceCode bash"><span class="kw">zcat</span> Test_2.fq.gz <span class="kw">|</span> <span class="kw">awk</span> <span class="st">&#39;{if(FNR%4==1) ID=$0; else if(FNR%4==2) seq_len=length($0); else if(FNR%4==0) {quality_len=length($0); if(seq_len!=quality_len) print ID; }}&#39;</span> </code></pre></div>
<ul>
<li>筛选差异基因</li>
</ul>
<div class="sourceCode"><pre class="sourceCode bash"><code class="sourceCode bash"><span class="kw">ct@ehbio</span>:~/sxbd$ cat de_gene
<span class="kw">ID</span>  log2fc  padj
<span class="kw">A</span>   1   0.001
<span class="kw">B</span>   -1  0.001
<span class="kw">C</span>   1   0.001
<span class="kw">D</span>   2   0.0001
<span class="kw">E</span>   -0.51   0.051
<span class="kw">F</span>   0.1 0.1
<span class="kw">G</span>   1   0.1

<span class="kw">ct@ehbio</span>:~/sxbd$ awk <span class="st">&#39;$3&lt;0.05 || NR==1&#39;</span> de_gene 
<span class="kw">ID</span>  log2fc  padj
<span class="kw">A</span>   1   0.001
<span class="kw">B</span>   -1  0.001
<span class="kw">C</span>   1   0.001
<span class="kw">D</span>   2   0.0001

<span class="kw">ct@ehbio</span>:~/sxbd$ awk <span class="st">&#39;BEGIN{OFS=FS=&quot;\t&quot;}{if(FNR==1) print $0; else {abs_log2fc=($2&lt;0?$2*(-1):$2);if(abs_log2fc&gt;=1 &amp;&amp; $3&lt;0.05) print $0;}}&#39;</span> de_gene 
<span class="kw">ID</span>  log2fc  padj
<span class="kw">A</span>   1   0.001
<span class="kw">B</span>   -1  0.001
<span class="kw">C</span>   1   0.001
<span class="kw">D</span>   2   0.0001</code></pre></div>
<ul>
<li>筛选差异基因存储到不同的文件</li>
</ul>
<div class="sourceCode"><pre class="sourceCode bash"><code class="sourceCode bash"><span class="kw">ct@ehbio</span>:~/sxbd$ awk <span class="st">&#39;BEGIN{OFS=FS=&quot;\t&quot;; up=&quot;up&quot;; dw=&quot;dw&quot;;}{if(FNR==1) {print $0 &gt;up; print $0 &gt;dw;} else if ($3&lt;0.05) {if ($2&gt;=1) print $0 &gt;up; else if($2&lt;=-1) print $0 &gt;dw;}}&#39;</span> de_gene 
<span class="kw">ct@ehbio</span>:~/sxbd$ head up dw
==<span class="kw">&gt;</span> <span class="kw">up</span> <span class="kw">&lt;</span>==
<span class="kw">ID</span>  log2fc  padj
<span class="kw">A</span>   1   0.001
<span class="kw">C</span>   1   0.001
<span class="kw">D</span>   2   0.0001

==<span class="kw">&gt;</span> <span class="kw">dw</span> <span class="kw">&lt;</span>==
<span class="kw">ID</span>  log2fc  padj
<span class="kw">B</span>   -1  0.001</code></pre></div>
<ul>
<li>ID map，常用于转换序列的ID、提取信息、合并信息等</li>
</ul>
<div class="sourceCode"><pre class="sourceCode bash"><code class="sourceCode bash"><span class="kw">ct@ehbio</span>:~/sxbd$ cat id_map 
<span class="kw">ENSM</span>    Symbol  Entrez
<span class="kw">ENSG00000280516</span> TMEM42  693149
<span class="kw">ENSG00000281886</span> TGM4    7047
<span class="kw">ENSG00000280873</span> DGKD    8527
<span class="kw">ENSG00000281244</span> ADAMTS13    11093
<span class="kw">ENSG00000280701</span> RP11-272D20.2   
<span class="kw">ENSG00000280674</span> ZDHHC3  51304
<span class="kw">ENSG00000281623</span> Y_RNA   
<span class="kw">ENSG00000280479</span> CACFD1  11094
<span class="kw">ENSG00000281165</span> SLC2A6  11182
<span class="kw">ENSG00000281879</span> ABO 28
<span class="kw">ENSG00000282873</span> BCL7A   605
<span class="kw">ENSG00000280651</span> AC156455.1  100506691
<span class="kw">ct@ehbio</span>:~/sxbd$ vim ensm
<span class="kw">ct@ehbio</span>:~/sxbd$ cat ensm 
<span class="kw">ENSG00000281244</span>
<span class="kw">ENSG00000281165</span>
<span class="kw">ENSG00000282873</span>
<span class="kw">ct@ehbio</span>:~/sxbd$ awk <span class="st">&#39;BEGIN{OFS=FS=&quot;\t&quot;}ARGIND==1{if(FNR&gt;1) ensm2entrez[$1]=$3;}ARGIND==2{print ensm2entrez[$1];}&#39;</span> id_map ensm
<span class="kw">11093</span>
<span class="kw">11182</span>
<span class="kw">605</span>

<span class="co"># 替代解决方案，注意 -w的使用，避免部分匹配。最稳妥的方式还是使用awk。</span>

<span class="kw">ct@ehbio</span>:~/sxbd$ grep -w -f ensm id_map <span class="kw">|</span> <span class="kw">cut</span> -f 3
<span class="kw">11093</span>
<span class="kw">11182</span>
<span class="kw">605</span></code></pre></div>
<ul>
<li>转换大小写, <code>toupper</code>, <code>tolower</code></li>
</ul>
<div class="sourceCode"><pre class="sourceCode bash"><code class="sourceCode bash"><span class="kw">ct@ehbio</span>:~/sxbd$ cat symbol 
<span class="kw">Tgm4</span>
<span class="kw">Dgkd</span>
<span class="kw">Abo</span>

<span class="kw">ct@ehbio</span>:~/sxbd$ awk <span class="st">&#39;BEGIN{OFS=FS=&quot;\t&quot;}ARGIND==1{if(FNR&gt;1) ensm2entrez[$2]=$3;}ARGIND==2{print ensm2entrez[toupper($1)];}&#39;</span> id_map symbol 
<span class="kw">7047</span>
<span class="kw">8527</span>
<span class="kw">28</span></code></pre></div>
<ul>
<li>awk数值操作</li>
</ul>
<div class="sourceCode"><pre class="sourceCode bash"><code class="sourceCode bash"><span class="co"># log2对数</span>
<span class="kw">awk</span> <span class="st">&#39;BEGIN{OFS=&quot;\t&quot;;FS=&quot;\t&quot;}{print log($0)/log(2)}&#39;</span> file

<span class="co"># 取整,四舍五入</span>
<span class="kw">awk</span> <span class="st">&#39;BEGIN{OFS=&quot;\t&quot;;FS=&quot;\t&quot;}{print int($1+0.5);}&#39;</span> file</code></pre></div>
<ul>
<li>awk定义函数</li>
</ul>
<div class="sourceCode"><pre class="sourceCode bash"><code class="sourceCode bash"><span class="kw">awk</span> <span class="st">&#39;function abs(x){return ((x &lt; 0.0) ? -x : x)}BEGIN{OFS=&quot;\t&quot;;FS=&quot;\t&quot;}{pos[1]=$1;pos[2]=$2;pos[3]=$3;pos[4]=$4; len=asort(pos);for(i=len;i&gt;1;i--) print abs(pos[i]-pos[i-1]);}&#39;</span> file</code></pre></div>
<ul>
<li>字符串匹配</li>
</ul>
<div class="sourceCode"><pre class="sourceCode bash"><code class="sourceCode bash"><span class="kw">ct@ehbio</span>:~/sxbd$ cat ens.bed
<span class="kw">1</span>   100 105
<span class="kw">2</span>   100 105
<span class="kw">3</span>   100 105
<span class="kw">Mt</span>  100 105
<span class="kw">X</span>   100 105
<span class="kw">ct@ehbio</span>:~/sxbd$ awk <span class="st">&#39;BEGIN{OFS=FS=&quot;\t&quot;}{if($1~/^[0-9XY]/) $1=&quot;chr&quot;$1; else if($1~/M.*/) gsub(/M.*/, &quot;chrM&quot;, $1); print $0}&#39;</span> ens.bed 
<span class="kw">chr1</span>    100 105
<span class="kw">chr2</span>    100 105
<span class="kw">chr3</span>    100 105
<span class="kw">chrM</span>    100 105
<span class="kw">chrX</span>    100 105</code></pre></div>
<ul>
<li>字符串分割</li>
</ul>
<div class="sourceCode"><pre class="sourceCode bash"><code class="sourceCode bash"><span class="kw">ct@ehbio</span>:~/sxbd$ cat trinity_id
<span class="kw">Trinity_C1_g1_i1</span>
<span class="kw">Trinity_C1_g1_i2</span>
<span class="kw">Trinity_C1_g1_i3</span>
<span class="kw">Trinity_C2_g1_i1</span>
<span class="kw">Trinity_C3_g1_i1</span>
<span class="kw">Trinity_C3_g3_i2</span>
<span class="kw">ct@ehbio</span>:~/sxbd$ awk <span class="st">&#39;BEGIN{OFS=FS=&quot;\t&quot;}{count=split($1, geneL, &quot;_&quot;); gene=geneL[1]; for(i=2;i&lt;count;i++) gene=gene&quot;_&quot;geneL[i]; print gene,$1;}&#39;</span> trinity_id 
<span class="kw">Trinity_C1_g1</span>   Trinity_C1_g1_i1
<span class="kw">Trinity_C1_g1</span>   Trinity_C1_g1_i2
<span class="kw">Trinity_C1_g1</span>   Trinity_C1_g1_i3
<span class="kw">Trinity_C2_g1</span>   Trinity_C2_g1_i1
<span class="kw">Trinity_C3_g1</span>   Trinity_C3_g1_i1
<span class="kw">Trinity_C3_g3</span>   Trinity_C3_g3_i2</code></pre></div>
<ul>
<li>awk脚本</li>
</ul>
<div class="sourceCode"><pre class="sourceCode bash"><code class="sourceCode bash"><span class="kw">cat</span> &lt;&lt;END &gt;grade.awk
f ( avg &gt;= 90 ) grade=&quot;A&quot;;
else if ( avg &gt;= 80) grade =&quot;B&quot;;
else if (avg &gt;= 70) grade =&quot;C&quot;;
else grade=&quot;D&quot;;
END

awk -f grade.awk grade</code></pre></div>
<ul>
<li>awk给每行增加行号，使其变为唯一</li>
</ul>
<div class="sourceCode"><pre class="sourceCode bash"><code class="sourceCode bash"><span class="kw">awk</span> <span class="st">&#39;BEGIN{OFS=&quot;\t&quot;;FS=&quot;\t&quot;}NR!=1{$4=$4&quot;_&quot;NR;print $0}&#39;</span> file</code></pre></div>
</div>
<div id="section-3.3" class="section level2">
<h2><span class="header-section-number">3.3</span> 糅合操作</h2>
<ul>
<li>awk中执行系统命令 (注意引号的使用)</li>
</ul>
<div class="sourceCode"><pre class="sourceCode bash"><code class="sourceCode bash"><span class="co"># 系统命令组成字符串，交给system函数运行</span>
<span class="kw">awk</span> <span class="st">&#39;BEGIN{OFS=FS=&quot;\t&quot;}{system(&quot;mv &quot;$1&quot;.fq &quot;$2&quot;.fq&quot;);}&#39;</span> input_mat</code></pre></div>
<ul>
<li>awk 引用系统变量</li>
</ul>
<div class="sourceCode"><pre class="sourceCode bash"><code class="sourceCode bash"><span class="kw">ct@ehbio</span>:~/sxbd$ echo 1 <span class="kw">|</span> <span class="kw">awk</span> -v ehbio=<span class="st">&quot;shengxinbaodian&quot;</span> -v ehbio2=<span class="st">&quot;SXBD&quot;</span> <span class="st">&#39;{print ehbio, ehbio2;}&#39;</span>
<span class="kw">shengxinbaodian</span> SXBD</code></pre></div>
<p>学会了基本命令，生信分析中还有一大块是使用已经安装好的工具，针对软件安装中遇到的问题，推出了系列文章：<a href="http://mp.weixin.qq.com/s?__biz=MzI5MTcwNjA4NQ==&amp;mid=2247483872&amp;idx=1&amp;sn=7fb7e57b3ff5c06ebaff344370c8b4c8&amp;chksm=ec0dc46adb7a4d7cc125ab3cf8361bf3e3fcf858edca7d7987d52bce3e9e3aa0b7d8f8f2adfa#rd">环境变量和可执行属性</a>彻底释义环境变量的概念；列举出Linux下<a href="http://mp.weixin.qq.com/s?__biz=MzI5MTcwNjA4NQ==&amp;mid=2247483954&amp;idx=1&amp;sn=11247591a6ef98a4d25404278d577ed0&amp;chksm=ec0dc7b8db7a4eaeb7ae3fd2fa2fbfa7bfd13f5e90d7a42d405f6f8e8783761de048f7ccbc58#rd">软件安装</a>的各种方法，并针对<a href="http://mp.weixin.qq.com/s?__biz=MzI5MTcwNjA4NQ==&amp;mid=2247483840&amp;idx=1&amp;sn=f87f6dd703cd8c109f6dc5b8d12ffb7c&amp;chksm=ec0dc44adb7a4d5c9ff2422c730b1d7bb18dcb6947c0e7449f1678aee492c3193302174930b4#rd">Docker</a>和<a href="http://mp.weixin.qq.com/s/A4_j8ZbyprMr1TT_wgisQQ">Conda</a>分别发文介绍。</p>
</div>
<div id="section-3.4" class="section level2">
<h2><span class="header-section-number">3.4</span> 生信宝典</h2>
<ul>
<li><a href="http://mp.weixin.qq.com/s/8wD14FXt7fLDo1BjJyT0ew">Linux学习 - 常用和不太常用的实用awk命令</a></li>
</ul>
</div>
<div id="rpythoncytoscape" class="section level2">
<h2><span class="header-section-number">3.5</span> 生信宝典，生物信息学习系列教程，转录组，宏基因组，外显子组，R作图，Python学习，Cytoscape视频教程</h2>
<p><a href="http://mp.weixin.qq.com/s/d1KCETQZ88yaOLGwAtpWYg" class="uri">http://mp.weixin.qq.com/s/d1KCETQZ88yaOLGwAtpWYg</a></p>
</div>
<div id="section-3.6" class="section level2">
<h2><span class="header-section-number">3.6</span> 生信宝典，最好的生物信息培训课程，培训课程资料</h2>
<p><a href="www.ehbio.com/Training" class="uri">www.ehbio.com/Training</a></p>

</div>
</div>
            </section>

          </div>
        </div>
      </div>
<a href="softinstall.html" class="navigation navigation-prev " aria-label="Previous page"><i class="fa fa-angle-left"></i></a>
<a href="sed.html" class="navigation navigation-next " aria-label="Next page"><i class="fa fa-angle-right"></i></a>
    </div>
  </div>
<script src="libs/gitbook-2.6.7/js/app.min.js"></script>
<script src="libs/gitbook-2.6.7/js/lunr.js"></script>
<script src="libs/gitbook-2.6.7/js/plugin-search.js"></script>
<script src="libs/gitbook-2.6.7/js/plugin-sharing.js"></script>
<script src="libs/gitbook-2.6.7/js/plugin-fontsettings.js"></script>
<script src="libs/gitbook-2.6.7/js/plugin-bookdown.js"></script>
<script src="libs/gitbook-2.6.7/js/jquery.highlight.js"></script>
<script>
gitbook.require(["gitbook"], function(gitbook) {
gitbook.start({
"sharing": {
"github": false,
"facebook": false,
"twitter": false,
"google": false,
"weibo": false,
"instapper": false,
"vk": false,
"all": ["facebook", "google", "twitter", "weibo", "instapaper"]
},
"fontsettings": {
"theme": "white",
"family": "sans",
"size": 2
},
"edit": {
"link": null,
"text": null
},
"download": ["Bash.pdf"],
"toc": {
"collapse": "subsection"
}
});
});
</script>

<!-- dynamically load mathjax for compatibility with self-contained -->
<script>
  (function () {
    var script = document.createElement("script");
    script.type = "text/javascript";
    script.src  = "https://cdn.bootcss.com/mathjax/2.7.1/MathJax.js?config=TeX-MML-AM_CHTML";
    if (location.protocol !== "file:" && /^https?:/.test(script.src))
      script.src  = script.src.replace(/^https?:/, '');
    document.getElementsByTagName("head")[0].appendChild(script);
  })();
</script>
</body>

</html>
